A phase diagram for a topological Kondo insulating system 
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The discovery of topological insulators in non-interacting electron systems has motivated the 
community to search such topological states of matter in correlated electrons both theoretically and 
experimentally. In this paper we investigate a phase diagram for a topological Kondo insulating 
system, where an emergent "spin" -dependent Kondo effect gives rise to an inversion for heavy- 
fermion bands, responsible for a topological Kondo insulator. Resorting to the U(l) slave-boson 
mean-field analysis, we uncover an additional phase transition inside the Kondo insulating state 
in two dimensions, which results from the appearance of the topological Kondo insulator. On the 
other hand, we observe that the Kondo insulating state is distinguished into three insulating phases 
in three dimensions, identified with the weak topological Kondo insulator, the strong topological 
Kondo insulator, and the normal Kondo insulator, respectively, and classified by Z2 topological 
indices. We discuss the possibility of novel quantum criticality between the fractionalized Fermi 
liquid and the topological Kondo insulator, where the band inversion occurs with the formation of 
the heavy-fermion band at the same time. 

PACS numbers: 



I. INTRODUCTION 

An effective field theory approach has been playing an 
important role in predicting novel quantum states of mat- 
ter. They are well developed chiral anomaly for quantum 
number fractionalization, 1 parity anomaly for quantum 
Hall effect^ and an SU(2) global anomaly 3 - for topolog- 
ical insulators in three dimensions*. However, the field 
theory approach is not enough to search such quantum 
matter in real materials. It is necessary to construct or 
find the corresponding lattice model, which can result in 
our wishful effective field theory at low energies. In par- 
ticular, the lattice model can be solved almost exactly 
based on numerical simulations, helping us understand 
the connection between the lattice model and effective 
field theory and verifying the possibility of novel quan- 
tum phenomena. 

The Su-Schrieffer-Heeger model, which describes one 
dimensional electrons coupled with lattices, confirms the 
existence of e/2 fractional electric charge, carried by do- 
main wall solitons^ The Haldane model, which describes 
two dimensional electrons with a next nearest neighbor 
complex hopping parameter on the honeycomb lattice, 
explains the integer quantum Hall effect without Lan- 
dau levels formed from external magnetic field^ where 
each Dirac band carries a nontrivial topological quan- 
tum number (Chern number), identified with the quan- 
tized Hall conductance^ Recently, the Haldane model 
has been generalized into the case with time reversal in- 
variance, where the spin-orbit coupling serves an effective 
magnetic flux, oppositely assigned to spin up and down 
electrons, thus regarded as two copies of Haldane mod- 
els. The Kane-Mele model has proposed an interesting 
insulating state, called a topological insulator, where the 
quantum spin Hall effect appears when the z-component 



of spin quantum number is preserved, but generically 
classified by the Z2 topological quantum number with 
spin non-conserving terms.* The concept of the topolog- 
ical insulator was extended into the three dimensional 
case^r— characterized by the Z2 topological index which 
counts the number of band inversions with modular two. 
Odd number of band inversions cause odd number of 
Dirac bands, identified with normalizable fermion zero 
modes localized at each two dimensional surface, where 
one of them at least is protected against time reversal 
invariant perturbations due to a topological origin. 

Considering that these models describe non-interacting 
electrons basically, an immediate and important ques- 
tion is on the role of electron correlations in such topo- 
logical states of matter.— Actually, this direction of re- 
search has been performed intensively, in particular, fo- 
cusing on the emergence of gapped spin liquids ^ where 
spinons with fractional spin quantum number 1/2 ap- 
pear as elementary excitations. The quantum dimer 
model on the triangular lattice has confirmed the exis- 
tence of a short-ranged resonating valence bond state, 
identified with Z2 spin liquid^ This gapped spin liq- 
uid state has been claimed to appear in strongly cor- 
related electrons on the honeycomb latticed Such a 
spirit has been also realized in strongly correlated spin- 
less fermionsji£ where charge fractionalization occurs in 
geometrically frustrated lattices. Artificial spin models 
but exactly solvable, represented as the Kitaev model, 17 
have been investigated, proving the existence of topologi- 
cally non-trivial quantum states of matter, for example, a 
non-abelian fractional quantum Hall state, which allows 
Majorana fermions.— Such exotica has been also pursued 
near quantum criticality, referred as deconfined quantum 
criticality, where an effective field theory approach had 
suggested the emergence of spin quantum number frac- 
tionalization at an antiferromagnet to valence bond solid 
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quantum critical point ^ and numerical simulations for 
an extended Heisenberg model on the square lattice con- 
firmed this scenario with some modifications . 20 ' 21 

Recently, the fractional quantum Hall effect has been 
argued to appear in a certain type of lattice models, 
whose characteristic feature is the existence of an almost 
flat band with a non-trivial Chern number^ Electron 
correlations for this partially filled flat band with the 
Chern number were demonstrated to result in the frac- 
tional quantum Hall phase. An immediate and interest- 
ing question is how to construct the lattice model, which 
shows the so called fractional topological insulator. An 
effective field theory approach argued the possibility of 
the fractional topological insulator ; 23 ' 24 characterized by 
the fact that the surface Dirac fermion carries a fractional 
electric charge, which should be distinguished from the 
fractionalized "normal" insulator, not allowing gapless 
surface modes. A hard-core boson model was proposed 
on the diamond lattice, where the boson is assumed to 
fractionalize into two fermions with e/2 fractional charge, 
forming a topological band insulator for such fractional- 
ized fermions and displaying the fractional magnetoelec- 
tric effect^ Introducing electron correlations into the 
Kane-Mele model, 26 '" 2 ? an interesting spin liquid state 
was proposed to exhibit an integer quantum spin Hall 
effect in the case of S z conservation. 

In this paper we investigate an effective Anderson lat- 
tice model for the so called topological Kondo insulator, 
regarded as one of the most studied models, particulary, 
for non-Fermi liquid physics near heavy fermion quan- 
tum criticality^ The existence of the topological Kondo 
insulator was pointed out recently, where an emergent 
"spin" -dependent Kondo effect gives rise to an inversion 
for heavy-fermion bands, which turns out to be respon- 
sible for the topological Kondo insulator^ Resorting to 
the U(I) slave-boson mean- field analysis, we uncover an 
additional phase transition inside the Kondo insulating 
state in two dimensions, which results from the appear- 
ance of the topological Kondo insulator. On the other 
hand, we observe that the Kondo insulating state is dis- 
tinguished into three insulating phases in three dimen- 
sions, identified with the weak topological Kondo insu- 
lator, the strong topological Kondo insulator, and the 
normal Kondo insulator, respectively, and classified by 
Z2 topological indices. Such phase transitions inside the 
Kondo insulator are described by gap closing. We derive 
an effective Dirac theory near the gap closing momentum 
point for the phase transition from the strong topological 
Kondo insulator to the normal Kondo insulator. 



II. PHASE DIAGRAM FOR A TOPOLOGICAL 
KONDO INSULATOR 

A. An effective Anderson lattice model 

In order to consider the topological nature of the heavy 
electron system, we construct the simplest model which 



introduces one s-orbital and one /-orbital in the unit cell. 
Conduction electrons are described by 

k a 

where e% is the dispersion relation of the conduction elec- 
tron with momentum k and real spin er, and fi c is the 
electron chemical potential. The /-electron Hamiltonian 
is given by the site energy e/— /i c and the on-site Coulomb 
interaction U, 

H f = EE( e / ~ VclfirfiT + U ^71 /ir 71 /i _ T ,(2) 
i r i 

where nfi T = f\ T fi T is the density operator of the /- 
electron with pseudo-spin r belonging to one representa- 
tion 7 of j = 5/2 multiplet at site i. \i — t) is the time 
reversal partner of \ir) in the Kramers doublet. Repre- 
sentations other than the 7 representation are assumed 
to be irrelevant. 

An essential point is how these itinerant and localized 
electrons are coupled, generically given by the hybridiza- 
tion term 

H hyb = E E V °r{k)c{j kT + H.c, (3) 

k a,T 

where fk T is the Fourier fc-component of fi T . In order 
to determine the structure of V aT (k), we have to specify 
the representation 7 of /-electron and its surroundings 
associated with conduction electron sites. If we restrict 
the /-electron states onto the j = 5/2 multiplet, the 
hopping matrix is described by 

(isa\H hyb \jjT) ~ {isa\H h y b \j-qa){jrjCT\jma) 

5 5 
x (jma\jj = = -/i|jV),(4) 

where rj is the representation and basis of the cubic har- 
monic oscillator, m is the z-component of the orbital an- 
gular momentum of /-electron, and fi is the z-component 
of the total angular momentum. The approximate equal- 
ity comes from the ignorance of the j = 7/2 multiplet in 
estimating the matrix element. 

Resorting to the table of (isa\Hh yb \jr]a) m Ref^, we 
can estimate s-f integrals as follows 

(is\H hyb \jxyz) = Vl5lmn(sfa), (5) 
(is\H hyb \jx(5x 2 - 3r 2 )) = ^(5Z 2 - 3)(s/a), (6) 

(is\H hyb \jx(y 2 - z 2 )) = ^Vl5l(m 2 - n 2 )(s/a), (7) 

where l,m,n are the direction cosine of the vector i — j. 
The factor of (ji]a\jma) corresponds to the weight of 
spherical harmonics in cubic harmonics. (jma\jj = 
is nothing but the Clebsch-Gordan coefficient. The last 
factor of (jj = |^| J7r) of Eq. (4) is the weight of \j = 
5/2/u) in 1 77"). Using the bases in the cubic crystalline 
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TABLE I: (ria\^T). r\ describes a cubic harmonic oscillator. 
The wave function of I77-) is chosen by those of j = 5/2 in the 
cubic crystal structure. 





r 7 + 


r 7 - 


Tg(i)+ r 8( i)— 


Ts(2)+ r 8 ( 2 ) — 


A 2u t 



















T iu at 











3 





vf- 


Ti„^t 








U 


3 ■ 

2V7 





2v /7 1 


Ti„7t 














/3 

V 7 





T 2u 7)t 






2 






2v / 2T 

275T 1 






2 V7 
%/5 • 




73T 





"TIT 











A 2u 4 





1 — 














TiuO I 
T lu /3i 










3 

2^/7 

"277 1 






2% /7 

_ VLi 

2^7 

























T 2u £ 1 


2 





V5 
2vT2T 










T 2u ?7| 







2V/2T 





2 V7 J 





T 2 „Ct 





2 





V5 
V21 









cess results from the nearest neighbor hopping from the 
/-electron state at a site i to the s-electron state at a 
neighboring site i + e, where e is a vector connecting with 
a neighboring unit cell. 

Using the hybridization matrix discussed above, we ob- 
tain the following matrix element for the hopping to the 
(100) direction, 



(i + zst \H hyb \iT 8{1) -) 
(i + xst \H hyb \iT 8{2) -) 



(i + xs I |i? h3/i ,|ir 8 ( 1 )+) 

(i + XS I \H hyb \iT s{2) +) 
V3 



2V7 



(9) 



for the hopping matrix element to (010) direction, 
(i + t/st \H hyb \ir sw -) = -(i + ys i\H hyh \iT s{1) +) 

' Ksfa), (10) 



2V7 



(i + ys t \H hyb \iT 8{2) -) = -(i + ys I \H hyb \iT 8{2) +) 



Vs. 



2V7 



(11) 



electric field for I77-), we obtain the table of (r/a \-fr) in 
Table I. 

Next, we specify the hybridization process as the or- 
bital of the conduction electron is situated at the same 
position like the case of 6s-electron states of rare earth 
ions. Then, there is no local hybridization between s- 
and /-electrons. Therefore, the main hybridization pro- 



for the hopping matrix element to (001) direction 
(i + zst \H hyb \iT 8{2) +) = 



i + zs I \H hyb \iT 8{2) -) 
3' 



(sfcr), 



(12) 



respectively. 



J 



Introducing V s j — ~0^{ s f a ) f° r simplicity, we obtain the hybridization Hamiltonian to (lOO)-direction as 



H 



(100) 

hyb 



E([4 



fet C ki 



2V3i sin k x 
2v / 3isin/c a; 



/fer 8(1) + 
fkr s 



8(1)" 



+ 



fet C ki 





— 2i sin k x 



— 2isin k x 




/fer 8(2) + 
fkr s(2) - 



)+H.c, 



(13) 



for (OlO)-direction as 



H, 



(010) 

hyb 



sf 



E([4t 4 



2v/3sinfcj, 
-2\/3sinfcj / 



/fcr 8( i) + 
/fer 8(1) - 



t „t 

fet ' /.:. 





-2 sin fc„ 



2 sin fc y 




for (OOl)-direction as 



(001) 

hyb 



v„ 



£[4 



fet c fet 



4i sin fc 2 






— 4i sin fc z 



/fcr 8 ( 2 ) + 
/fer s(2 ) - 



/fcr 8 ( 2 ) + 
/fcr 8 



-8(2)- 



+ JT.c 



Gathering all these terms, the resulting hybridization Hamiltonian is given by 



Hhyb = V s f 



E[4 



fef 4t 



Vr 8(1) (fc) 



/fer 8(1) + 
/fer 8( i)- 



fcr 8 



// 
/fcr, 



(2)" 



(14) 



(15) 



(16) 
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where the hybridization matrix 

Vr(fe) = d r {k) ■ & (17) 

is 

dr g(1) (fc) = {2\/3ismk x , 2v / 3isinfc I/ , 0), (18) 
dr s(2) (fe) = (— 2isin k x , 2isin £; y , 4i sin k z ). (19) 



We note that this model Hamiltonian describes the 
nearest neighbor hybridization process between s- and 
/-electrons of rare earth ions in the simple cubic lattice. 
In appendix A we discuss how the hybridization Hamil- 
tonian is modified when the lattice symmetry is lowered 
from the simple cubic to the tetragonal. 



B. U(l) slave-boson mean-field analysis 



We start from an effective Anderson lattice model 

H = E E^ - Mc)cL C fe* + E E^/ ~ Mc)/ia/<a + e E E tij^'fljja* + ^ E E fiJitfi-Ji-" 
ho i a ij act' i a 

+t/ EEEt dr ( fc ) ' **°<Lfi«e~ ih ' rt +H.c], (20) 

k i acr 

where tij aa i is the hopping integral of /-electrons with a parameter t < 1 and the subscript sf in V s f is omitted 
for simplicity. This effective hopping for localized fermions is introduced phenomenologically to describe possible 
phase transitions from the "fractionalized" Fermi liquid stated to the Kondo insulating phase at zero temperature, 
which can be regarded to result from Ruderman-Kittel-Kasuya-Yoshida (RKKY) spin correlations effectively in an 
intermediate energy scaled The hybridization coupling term is given by Eqs. (18) and (19) for the simple cubic 
case £2- 

We take the U(l) slave-boson representation for the Kondo effect in the strong coupling limit, where the localized 
fermion is expressed as fi a — » b\fi a with the single occupancy constraint b\bi + f} a fia = 1 • Then, the effective 

a-± 

Lagrangian is given by 

L = E E c L(^ - + ei) Cka + vj2J2&™W c l* b lf™ e ~ ikri + h. c .\ 

+ E E fLfr +tf-vc+ iXi)h« - e E E w/JaV**' + 1 J2 Xiibtbi - 1), (21) 

i ct ij an' i 

where Aj is the Lagrange multiplier field for the constraint and & aa (k) = dr(fe) • <Taa- 

Taking bi — > b = (bi) with i\i — > A in the saddle-point approximation, we obtain two self-consistent equations for b 
and A, given by 

h aa'a K 

b 2 -~J2J2 dun F (u)ha.Gf ux (u + iO + ,k) = l, (23) 

n k a "'- 00 

respectively. G^(ui,k) is the spinon Green's function, given by 

G / (w,fe)= (uj-e f +Hc~\)l + et(k)-V 2 b 2 1 '— 1 ' , (24) 
L oj - e c k + [i c J 



where [t(k)] aa ' = X)ijlAj]aa' exp[ife- (rj — Tj)]. tif{uj) = C. Phase diagram 

l/(exp(w/T) + 1) is the Fermi-Dirac distribution func- 
tion. 

It is natural to expect a phase transition from the frac- 
tionalized Fermi liquid to the Kondo insulating state, de- 
scribed by the emergence of the Kondo effect (b ^ 0) 
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above a critical strength of hybridization. An interest- 
ing aspect of this effective lattice model is that there can 
exist additional phase transitions inside the Kondo in- 
sulating phase, not described by the holon condensation 
but characterized by the change of the Z 2 topological in- 
dex. 

For simplicity we take t aa i{k) = S aa d(k) with t(k) = 
e c k . Introduction of the spin dependence will not change 
possible phases but modify critical values of V associated 
with their phase transitions. The dispersion relations for 
the heavy fermion bands are given by 

E±{k) = -(4 - et(k) + € f + A) - /i c 



± 



^(4 + ri(fe)- e/ -A)H47W(fe), (25) 



respectively, where A 2 (/c) = iTr[<fr(fc) • <fr^(fc)]. 

In order to see how the band inversion occurs from this 
dispersion relation, we consider time reversal invariant 
momenta which satisfy k m = —k* m + G, where G is a re- 
ciprocal lattice vector. It is straightforward to check that 
there are eight time reversal invariant momentum points 
in three dimensions while four in two dimensions. The Z 2 
topological index, which measures how many times bands 
are twisted with modular two, has been reformulated for 
such time reversal invariant momenta in the case when 
the system preserves the inversion symmetry.— ~— First, 
we observe that the band gap in the Kondo insulating 
phase closes linearly at some of the time reversal symme- 
try points which satisfy e/ + A = £?. + et(k* n ), where 

A(fe) ~ |fe| appears near k* m . This identification leads us 
to define the parity S m = sign(e^,* + et{k* n ) — ej — A). 
See appendix B. Following Dzero et al^ we can evaluate 
the Z 2 topological indices given by 



I 



STI 



IK 



Hk' m ) a =0 



(26) 
(27) 



where Isti is an index for a strong topological insulator 
while Iwti are indices for a weak topological insulator. 
We will see these indices changed, showing additional 
phase transitions inside the Kondo insulator. 

We solve the mean field equations (22) and (23) with 
(24) for the simple cubic lattice with 



t(k) = — 2t(cos k x + cos k y + cos k z 



(28) 



numerically, and take the hybridization term of the r 8 ( 2 ) 
symmetry. Our slave-boson analysis uncovers 4 differ- 
ent phases in three dimensions, shown in Fig [T] When 
the hybridization coupling is smaller than a critical value 
V c , the Kondo effect does not exist, i.e., b — (Phase I), 
giving rise to local magnetic moments decoupled from 
conduction electrons, where such localized spins form a 
spinon "Fermi" liquid state (spin liquid) due to their dis- 
persions originating from the RKKY interactional Such 



an exotic liquid state may be realized in geometrically 
frustrated lattices or at finite temperatures <2i Increasing 
the hybridization coupling above V c , the Kondo effect re- 
sults in the formation of the heavy fermion band (6^0), 
but the condition of half filling for conduction electrons 
leads to an insulating state instead of the heavy fermion 
metal. An interesting point is that the Kondo insulating 
state is distinguished into three insulating phases, classi- 
fied by the Z 2 topological indices. The phase II is char- 
acterized by the trivial strong topological-insulator index 
of Isti = 1, but nontrivial weak topological-insulator in- 
dices of IyyTi = Iwti = ^wti = — 1< Thus, the phase II 
is identified with the weak topological Kondo insulator. 
The phase III is characterized by Isti — — 1 and I^ T 7 = 
ly/Ti ~ Iwti = 1- As a result, it is the strong topolog- 
ical Kondo insulator. The last Kondo insulating phase, 
phase IV, is the conventional Kondo insulator with trivial 
Z2 indices of Isti 



I and I^r TI 



1 WTI 



L WTI 



1. 



Since the first phase transition from the fractionalized 
Fermi liquid to the weak topological Kondo insulator is 




FIG. 1: U(l) slave-boson mean-field phase diagram for an ef- 
fective Anderson lattice model with the r 8 ( 2 ) hybridization on 
the simple cubic lattice. Phase I is the fractionalized Fermi 
liquid state, where localized spins are decoupled from conduc- 
tion electrons (b — 0), forming a spinon "Fermi" liquid state 
(spin liquid). The Kondo insulating phase (b 7^ 0) is dis- 
tinguished into Phase II, Phase III, and Phase IV, classified 
by the topological Z2 indices. Phase II is the weak topolog- 
ical Kondo insulator and Phase III is the strong topological 
Kondo insulator. Phase IV is the normal Kondo insulator. 
The first transition belongs to the second order in the saddle- 
point approximation, but fluctuation corrections are possi- 
ble to turn the nature of the continuous transition into the 
first order. On the other hand, phase transitions inside the 
Kondo insulator are also continuous, but regarded to be ro- 
bust against quantum corrections, where both hybridization 
and gauge fluctuations are irrelevant. The topological aspect 
of the band structure is changed by the gap closing transition. 
Model parameters of t = 1, e = 0.01, e/ = —6, and fi c = are 
used. 
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12 3 4 



FIG. 2: U(l) slave-boson mean- field phase diagram for an ef- 
fective Anderson lattice model with the rs(i) hybridization 
and k z = 0, i.e., on the square lattice. Phase I is the fraction- 
alized Fermi liquid state (6 = 0), Phase II is the topological 
Kondo insulator (6/0 and I^ti = ~ 1); an d Phase III is 
the normal Kondo insulator (6/0 and Iwti = !)■ Model 
parameters of t = 1, e = 0.01, e/ = —6, and fj, c — are used. 



FIG. 3: Top: The band structure in the Kondo insulating 
state (Phase III). It is almost identical with that in the topo- 
logical Kondo insulating phase (Phase II). The electron chem- 
ical potential lies between the band gap. Down: The band 
structure at the critical point inside the Kondo insulator. Gap 
closing appears at fe* w = (n, it), described by the Dirac theory 
from Eq. (31) to Eq. (39) with k z = 0. Model parameters of 
t = 1, £ = 0.01, and e/ = — 6 are used. 

given by the holon condensation, i.e., the formation of 



H = 



H{k) = 



where = (cj^., c^, f^ + ) is the four component 
Dirac spinor with e/ = e/ + A, and $(k) = 2\/3i sin k x + 
2v / 3sinfc y is the form factor of the r 8 (!) hybridization. 

Expanding H(k) around the gap closing point with 
time reversal symmetry, fc* TTr = (7r,7r,7r), we obtain an 



the Kondo effect, it belongs to the second order transi- 
tion in the saddle-point approximation. Two more phase 
transitions inside the Kondo insulator are described by 
gap closing, thus also belonging to the continuous transi- 
tion. An important point is that such second order tran- 
sitions inside the Kondo insulating phase are expected to 
be robust because singular corrections from fluctuations 
will not exist inside the Kondo insulator. In particular, 
both hybridization and gauge fluctuations will not play 
an important role in these phase transitions. On the 
other hand, such quantum corrections may be important 
for the nature of the first phase transition from the frac- 
tionalized Fermi liquid to the weak topological Kondo 
insulator, which will be discussed in the last section. 

It is straightforward to check the two dimensional 
case, i.e., the effective Anderson lattice model on the 
square lattice. Indeed, we performed the same slave- 
boson mean-field analysis in two dimensions and found 
the corresponding phase diagram of Fig. 2. The main 
difference from the three dimensional case is the absence 
of the strong topological Kondo insulating phase, where 
the quantum "spin" Hall Kondo insulating phase exists 
between the fractionalized Fermi liquid and the normal 
Kondo insulator. Fig. 3 display the band structures of 
both the topological (normal) Kondo insulating phase 
and the critical point, where the gap closing occurs at 



D. An effective Dirac theory for the hybridization 
coupling with the symmetry Ps(i) 

It is given by the Dirac theory an effective field the- 
ory for the phase transition from the strong topolog- 
ical Kondo insulator to the normal Kondo insulator. 
The effective slave-boson Hamiltonian for the topologi- 
cal Kondo insulator is 



(29) 

^ 

-Vb$*{k) ' ( 30 ) 
-et(k) + e f -ii c ) 

I 

effective Dirac theory 



^) = ( M o P) ^( -,))' (31) 



( £ C k -^c Vb^(k) 

Vb<P*(k) -et(k)+e f - [i c 

4 - Mc 

-Vb$(k) 
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where h(p) — e(p)ao + d(p) ■ cr is the two by two Dirac 
Hamiltonian with 



e(p) 


= C - D(v 2 + p 2 + p 2 ), 

^ \Jrx 1 iy 1 trzn 


(32) 


d{p) 


= (A Py ,A Px ,M(p)), 


(33) 


M(p) 


= M-B(pl+pl+p 2 z ), 


(34) 


G 


3 1„ 
= 3*+-e+-e/ -fi c , 


(35) 


D 


= 2<* + 2>' 


(36) 


A 


= -2V3Vb, 


(37) 


D 


= 2 ( ^2 } ' 


(38) 




3 1_ 


(39) 


M 


2 2 J 



This effective Hamiltonian turns out to be identical with 
that of Refi2£ when p x — > p y and p y — > p x are performed 
with p z = 0. The spectrum is 

E± (p) =s(p)± yjA*(j%+pl) + M*(p). (40) 

Note that sign(M) = Sk* corresponds to the parity 
at the symmetry point fc*^ = (tt,tt,tt). Recall that 
the Chern number is given by C = 1 for M/B > and 
C = for M/B < in the two dimensional case. This 
means that the parity 5k* describes the transition from 
the topological insulator to the trivial insulator via gap 
closing. 

In appendix C we derive an effective Dirac theory for 
the case of hybridization with r 8 ( 2 ). 

III. SUMMARY AND DISCUSSION 

It is essential to construct realistic lattice models in 
searching novel quantum states of matter. In this study 
we construct an effective Anderson lattice model in order 
to study an interplay between the topological aspect and 
strong correlation, regarded to be two cornerstones for 
novel quantum states of matter. The topological struc- 
ture could be introduced in the spin-pseudospin depen- 
dent hybridization between conduction electrons and lo- 
calized electrons, which originates from the interplay be- 
tween the spin-orbit interaction and crystalline electric 
field for localized /-electrons. The spin-pseudospin de- 
pendent Kondo effect plays essentially the same role as 
the spin-orbit coupling for topological insulators, allow- 
ing the topological Kondo insulator inside the Kondo 
insulating phase. Although its existence was already 
pointed out in an interesting recent study2£, our detailed 
slave-boson analysis has clarified its position in the phase 
diagram. In addition, we can argue that the existence of 
the topological Kondo insulating phase is robust against 
quantum corrections because hybridization fluctuations 
cannot be strong inside the Kondo insulator and gauge 
fluctuations are not, either. 



Unfortunately, it is difficult to say that such topolog- 
ical states are quite interesting because the appearance 
of the topological Kondo insulator just comes from the 
inversion of heavy fermion bands, basically the same as 
that in topological insulators of non-interacting electrons. 
However, we would like to point out that the present ef- 
fective lattice model has huge potential for novel quan- 
tum states of matter. In particular, we expect interest- 
ing spin-ordering structures in the position of the frac- 
tionalized Fermi liquid state, which results from the un- 
derestimation of spin correlations in the slave-boson ap- 
proach. In this respect the saddle-point analysis based 
on the slave-fermion theory^ will open the possibility of 
fruitful spin structures, where exotic ordering of local- 
ized spins may appear as a result of the spin-dependent 
Kondo effect. Recently, the integer quantum Hall effect 
was proposed in the ferromagnetically Kondo coupled lat- 
tice model on geometrically frustrated lattices^ where 
the kinetic-energy cost for conduction electrons becomes 
reduced due to the emergence of an internal magnetic 
flux from the formation of the spin chirality order. In 
addition, it is natural to expect the possibility of novel 
quantum criticality between the fractionalized Fermi liq- 
uid and the topological Kondo insulator because the band 
inversion occurs with the formation of the heavy-fermion 
band at the same time, quite uncommon in the Landau- 
Ginzburg- Wilson description for phase transitions. Of 
course, anomalous scaling near this quantum criticality 
is expected to be beyond the present mean-field analy- 
sis. Recently, two of us constructed an Eliashberg theory 
for the spin density wave transition in the surface state 
of the three dimensional topological insulator—, where 
the band reconstruction is not introduced but fluctuation 
corrections are incorporated. In this study we uncovered 
that the anomalous self-energy correction (the off diago- 
nal self-energy in the spin or pseudospin space) is essen- 
tial for self-consistency. We speculate that mathemat- 
ically the same self-energy correction via hybridization 
fluctuations will play an important role for this nontriv- 
ial quantum criticality. In particular, we expect that the 
characteristic feature for this quantum critical point may 
be introduced in scaling of the anomalous Hall conduc- 
tivity. Numerical simulations for this model seem to be 
invaluable. 
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Appendix A: Model Hamiltonian in tetragonal 
surroundings 

We consider the effect of symmetry lowering from cubic 
to tetragonal. In the cubic crystal structure, the level 
scheme of the /-electron for the j = 5/2 multiplet is 
given by 



c 5/2 1 o h = r 7 © r 8 , 



(Al) 



which serves bases for the extraction of the hybridization 
Hamiltonian. In the tetragonal surroundings, the Tj{Oh) 
and Ts(Oh) bases are reduced, 



r 7 (o h ) i D ih = r 7 , 

Ta(i)(Oh) I Dih = r 7 , 
T8(2)(0/i) 1 = r 6 , 



(A2) 
(A3) 
(A4) 



respectively. Therefore, Tj(Oh) and r 8 m(0/,) states can 
mix in the tetragonal surroundings, while r 8 ( 2 )(0/ 1 ) dou- 
blet splits from Ts(Oh) quartet by the tetragonal crys- 
talline electric field. 

Resorting to the above discussion, we can imagine a 
special case that only one /-electron doublet is relevant to 
describe the low energy physics in the tetragonal system. 
If the relevant doublet belongs to r 7 in the tetragonal 
system, the effective Hamiltonian will be 



=H C + H f +H r h ; b , 



(A5) 
(A6) 



H c = - Vc)c{ a C k<J , 

k a 

Hf=Y^ J2( e f ~ t 1 ") ft T fir + U^2n fiT n fi _ T (,A7) 



< 7 6 = E[4 t c^]dr T (k)-& 



fk+ 
fk- 



H.&8) 



with 



Sf = e f r r , (A9) 
d r7 (k) = (2y/SiV s f sin/c a; ,2v / 3iV;. / sinfc y ,0)(A10) 

fkr = /fcr 7 r, (All) 

where e/r 7 is the relevant r 7 doublet corresponding to 
fkr 7 r, and V s f is affected from the original V s f by the 
diagonalization in the tetragonal surroundings. 

When the relevant doublet is Tq, the effective Hamil- 
tonian becomes 



H Fe =H C + H f + H r ' 



hybt 



(A12) 
(A13) 



k a 

H f = J2J2( £ f ~ ^ftrfir + UJ2 nfirUfUA^) 



< 6 . = E[4 t 4 ]*.(*)•* 



fk+ 
fk- 



/i(A15) 



with 

Sf = £ f r e , (A16) 
dr 6 (fc) = (-2iV sf sin k x ,2iV sf sin k v ,iiVi f sin /(A)17) 

fkr = /fcr 6 r, (A18) 

where corresponding to the hybridization to z- 

direction is modified by the tetragonal surroundings. 
Here, s k is the dispersion relation of conduction electrons 
in the tetragonal surroundings. 



Appendix B: Review on the Z2 topological index 

We review the parity eigenvalue in the Z2 topological 
indices of Eqs. (26) and (27). Generally speaking, any 
four by four matrices with the hcrmitian property can be 
decomposed by the identity matrix J, five Dirac matri- 
ces T a , and their ten commutators r ab = [r Q , r b ]/(2i). 
Thus, our Hamiltonian matrix H T (k) can be expressed 
in terms of these 16 basis matrices. It is convenient to 
choose the following representation for the Dirac matri- 
ces, 



-1(1,2,3,4,5) 



(t 2 



1 (Tq,T x ® (To,T 



V 



where t q and cr a are 2x2 matrices in the orbital (s and 
/) and spin spaces, respectively. 

An essential aspect is that this general expansion can 
be simplified near gap closing momentum points, which 
occur at time reversal invariant momentum points fc* n , 
where the gap closes linearly in momentum, thus the ef- 
fective theory is given by the Dirac theory. Such a Dirac 
theory, referred as the Bernevig-Hughes-Zhang model^ 
can be expanded by only I and T a as 



Hl.Jk)^d r (k)I + J2d r a (k)T a . 



(B2) 



a=l 



Among these Dirac matrices, T 1 is nothing but the parity 
operator P in the present representation, 



Co, 



(B3) 



where t z — + and t z — — correspond to s- and /- 
orbitals, respectively. It should be noted the following 
relations on parity 

pi? a p 1 = r a 



pj? a p = r a 



for a = 1, (B4) 

for o/l. (B5) 

Therefore, T 1 is only parity-even Dirac operator. 

For the three dimensional system, there are eight time 
reversal invariant points k* n in the first Brillouin zone. 
We obtain two invariants at such time reversal invariant 
points, given by 

©H 1 ^)©- 1 = H r (k* m ), (B6) 
PH^k^p- 1 =H T {k; n ), (B7) 
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where is the time reversal operator. Considering the 
relation of Dirac matrices on parity transformation, we 
can easily estimate the eigenvalue at k* m 

E T {k* m ) = dl{k* m )I + d\{k* m )P, (B8) 

where d\(k* m ) is given by 

d r 1 (k* m ) = l(e c k . m +et(k* m )-e f -X). (B9) 
Then, the parity eigenvalue S m at k* m can expressed as 



follows 

5 m = -sgn(d{{k* m )) = -sgn(e%* m + et{k* m ) - e f - A). 

(BIO) 

Appendix C: An effective Dirac theory for the 
hybridization coupling with the symmetry Tg(2) 

In the hybridization with r 8 ( 2 ) the effective Hamilto- 
nian is 



H(k) 



Vb(2i sin k x + 2 sin k y ) 


—Wbi sin k z 



Vb(—2i sin k x + 2 sin k y ) 
-et(k) + e f - /j, c 
—Wbi sin k z 






Wbi sin k z 

£ k-^c 
-Vb(—2ismk x + 2smk y ) 



AVbi sin k z 


-Vb(2i sin k x + 2 sin k y ) 
-et(k) +€f - n c 



(CI) 



Expanding this Hamiltonian around the k^ vv point, 
we reach the following expression for the Dirac theory 



where 



H(p) 



Hp) 
g(p) 

e{p) 

d(p) 

M(p) 



Hp) aip) 

g*(p) h*(-p) 



e{p)a + d(p) ■ cr 
-2Aip z a x , 
C-D(pl+pl + 



Pi), 



(A Py ,-A Px ,M(p)) 
M-B{pl+p 2 y +p 



I), 



(C2) 



(C3) 
(C4) 
(C5) 
(C6) 
(C7) 



C = 3t 
1 



-e f - H c 



D 
A 



2 (< + l } ' 
2Vb, 



B = - 2 (t 



5' 

3 1„ 



M = 3i- -e- -e f . 

2 2 J 



(C8) 

(C9) 
(CIO) 
(Cll) 

(C12) 
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